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Abstract 

Pattern avoidance classes of permutations that cannot be expressed as 
unions of proper subclasses can be described as the set of subpermutations 
of a single bijection. In the case that this bijection is a permutation of 
the natural numbers a structure theorem is given. The structure theorem 
shows that the class is almost closed under direct sums or has a rational 
generating function. 
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1 Introduction 

Classes of permutations defined by their avoiding a given set of permutation 
patterns have been intensively studied within the last decade. Quite often the 
issue has been to determine the number of permutations of each length in the 
class. In order to do this it is necessary to derive structural properties of the 
permutations in the class starting from the avoided set. However, there are very 
few general techniques for obtaining such structural information. This paper is 
a contribution towards a general structure theory. We begin from the point of 
view that pattern-avoidance classes can be expressed as unions of atomic classes 
(those that have no non-trivial expression as a union) . We shall show that these 
atomic classes are precisely the classes that arise as the set of restrictions of 
some injection from one ordered set to another. In general the order types of 
these two sets provide some information about the atomic class. The major part 
of our paper is a characterisation of such injections and classes in the simplest 
case: when the order types are those of the natural numbers. 
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In the remainder of this section we review the terminology of pattern avoidance 
classes. Most of this terminology is standard in the subject except for the notion 
of a natural class. We shall see a number of conditions on pattern avoidance 
classes that are equivalent to their being atomic and we shall exhibit examples 
of atomic and non-atomic classes. These conditions and examples motivate the 
notion of a natural class whose elementary properties we explore in Section |21 
Section O contains our main result: a characterisation of natural classes, and 
Section 0] gives some further examples. 

We need a small number of definitions concerned with permutations and sets 
of permutations. For our purposes a permutation is just an arrangement of the 
numbers 1, 2, . . . , n for some n. Wc shall often need to consider arrangements of 
other sets of numbers and we shall refer to these as sequences; so, unless stated 
otherwise, a sequence will mean a list of distinct numbers. 

Two finite sequences of the same length a = 010203 • • ■ and (3 = &162&3 • ■ ■ 
are said to be order isomorphic (denoted as a = /?) if for all i,j we have 
ai < Uj if and only if bi < bj . Any sequence defines a unique order isomorphic 
permutation; for example 7496 = 3142. 

A sequence a is said to be involved in a sequence /3 (denoted as a ^ /3) if a is 
order isomorphic to a subsequence of f3. Usually, involvement is defined between 
permutations; for example 1324 ^ 146325 because of the subsequence 1435. 

It is easily seen that the involvement relation is a partial order on the set of 
all (finite) permutations. We study it in terms of its order ideals which we call 
closed sets. A closed set X of permutations has the defining property that if 
a G X and 5 ^ a then S E X. 

Closed sets are most frequently specified by their basis: the set of permutations 
that are minimal subject to not lying in the closed set. Once the basis B is 
given the closed set is simply 

{ct I /3 2< (T for aU (3 e B} 

and we shall denote it by A{B). 

Closed sets arise in the context of limited capability sorting machines such as 
networks of stacks, queues and deques with a point of input and a similar output. 
Here the basis consists of minimal sequences that cannot be sorted into some 
desirable order. As sequences can be sorted if and only if they do not involve any 
basis elements, basis elements are frequently referred to as "forbidden patterns" 
and the set of permutations that can be sorted by such a mechanism is described 
as the set of all permutations that "avoid" that basis. 

Much of the thrust of this paper is in specifying closed sets in a different way. 
Suppose that A and B are sets of real numbers and let tt be an injection from 
A to B. Then every finite subset {ci, C2, . . . , c„} of A, where ci < C2 < . . . < c„ 
maps to a sequence 7r(ci)7r(c2) ... 7r(c„) which is order isomorphic to some 
permutation. The set of permutations that arise in this way is easily seen to 
be closed and we denote it by Sub(7r : A ~> B). In many cases the domain and 
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range of tt are evident from the context in which case we write simply Sub(7r). 
Also, since we may always replace B by the range of tt we shall, from now on, 
assume that tt is a bijection. 



Example 1.1 Let A = {I - 1/2', 2 - 1/2* | i = 1, 2, . . .} and B = {1,2, . . .}. 
Let TT be defined by: 



Then it is easily seen that any finite increasing sequence of elements in A maps to 
an increasing sequence of odd integers followed by an increasing sequence of even 
integers. From this it follows readily that the permutations of Sub(7r : A B) 
are precisely those that consist of two increasing segments. As shown in this 
closed set has basis {321,3142,2143}. Notice that A and B have order types 
2uj and lo. This particular closed set cannot be defined as Sub(7r : A B) with 
both A and B having order type u. 

We now give several conditions on a closed set equivalent to it being expressible 
as Sub(7r -.A^B). 

Theorem 1.2 The following conditions on a closed set X are equivalent: 

1. X = Sub(7r : A ^ B) for some sets A, B and bijection n. 

2. X cannot be expressed as a union of two proper closed subsets. 

3. For every a, f3 Cz X there exists "f (z X such that a ^ 7 and f3 ^ ^. 

4-. X contains permutations 71 ^ 72 ■ ■ ■ such that, for every a £ X , we 
have a ^ jn for some n. 



1 2. Suppose X = Sub(7r : A B) and yet there exist proper closed subsets 
Y, Z oi X such that X ^ Y U Z. Then there exist permutations p G X \ Y and 
a € X\Z. Therefore we can find subsequences ri < r2 < • • • and si < S2 < ■ ■ ■ 
of A which are mapped by tt to subsequences order isomorphic to p and a. The 
union of {ri, r2, . . .} with {si, S2, . . .} defines a sequence ti < t2 < ■ ■ ■ that 
is mapped by tt to a subsequence order isomorphic to a permutation r 6 X. 
Obviously, p ^ t and cr ^ r. However r belongs to at least one of Y or Z, say 
T £ Y. Since X is closed wc have p G Y, a, contradiction. 

2 3. Suppose that there exist a, (3 £ X with the property that no permutation 
of X involves both of them. Put 




2i-l if a; = 1-1/2* 
2i ifa; = 2-l/2* 



Proof: 



Y = {7 e X I a ^ 7} 
Z = {7eX|/3^7} 
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Then Y and Z are proper closed subsets of X whose union is X (since any 
7 e X \ (y U Z) would involve both a and f3). 

3 4. If 0, are two permutations in X we know that there exists a permu- 
tation of X that involves both. Temporarily we shall use the notation 9 W (j> 
to denote one of these permutations. Now let /3i,/32, ■ • ■ be any listing of the 
permutations of X. We define a sequence of permutations of X as follows: 
7i = f3i and; for i > 2, 7^ = Ji-i V f3i. Obviously, 71 r< 72 ^ • • ■ and, for each 
permutation Pn ^ X , [3n In- 

4 1. In the sequence 71 ^ 72 ^ • • • we remove duplicates (if any) and we 
insert suitable permutations so that we have one of every degree. This gives a 
sequence of permutations ai < a2 ■ ■ ■ such that 

1. |aj| = i, 

2. Qfj G X, 

3. for all CT e X there exists some with <t < Ui 

Now we shall inductively define, for each i, sets Ai, Bi and bijections tt^ : — > 
Bi with the following properties: 

1. I A, I = \B,\=i 

2. TTi is order isomorphic to Ui 

3. Ai_i C and C A,; 

Once these sets have been constructed we can complete the proof by setting 
A = Uj and B = [J^Bi. Then we define tt : A ^ B for any a £ A by finding 
some Ai for which a G A^ and setting 7r(a) = 7r.t(a); by the last two properties 
TT is well-defined. The second property guarantees that X ~ Sub(7r : A ^ B). 

To carry out the construction we shall define Ai , Bi as subsets of the open 
interval (0, 1). We begin by setting Ai = = 1/2 and 7ri(l/2) = 1/2. Suppose 
now that Ai,Bi, tt^ have been constructed for i ~ 1,2 ... ,n. The permutation 
a„+i is constructed from by the insertion of a new element t at position s 
in a„; the position numbers of all the elements of a„ which are greater than or 
equal to s have to be increased by 1 and those values which are greater than or 
equal to t have also to be incremented by 1. 

We reflect this insertion in the definition of A„-|_i, Bn+i and 7r„_|_i. The set A„-|_i 
is formed by augmenting A„ with another number a whose value lies between 
its (s — 1)*'' and s*'' elements (if s = 1 we take a between and the minimal 
element of A„; while if s = n we take a between the maximal element and 1). 
Similarly i3„+i is formed by augmenting i?„ with a number b whose value lies 
between its {t — 1)*'' and t*^ elements. Then we define 7r„+i so that it agrees 
with 7r„ on the elements of A„ and has 7r„-|_i(a) — b. M 
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Because of this result we call closed sets of the form Sub(7r : A B) atomic on 
the grounds that they cannot be decomposed as a proper union of two closed 
subsets. Expressing a given closed set as a union of atomic sets is often very 
useful in discovering structural information. 

Example 1.3 (See [2]) ^(321, 2143) = ^(321, 2143, 3142) U yl(321, 2143, 2413) 

Given an arbitrary closed subset one might hope to find its properties by first 
expressing it as a union of atomic sets, and then discovering properties of the 
bijection tt associated with each atomic subset. Many difficulties impede this 
approach. It may happen that a closed set cannot be expressed as a finite union 
of atomic subsets. Moreover an atomic closed set may have a defining bijection 
TT whose domain and range have high ordinal type; in that case one might be 
hopeful that properties of these ordinals (in particular, limit points) might imply 
properties of X = Sub(7r : A ^ B). Despite this hope it seems sensible to begin 
the systematic study of atomic sets by looking at the case where the ordinal 
type of both A and B is that of the natural numbers N. 

2 Natural classes and sum-complete classes 

A natural class is a closed set of the form Sub(7r : N N). In other words, 
starting from a permutation tt of the natural numbers, we form all the finite 
subsequences of 7r(l),7r(2),... and define a natural class as consisting of the 
permutations order isomorphic to these subsequences. From now on we shall 
use the notation Sub(7r) (suppressing a notational reference to the domain and 
range of tt) in the following circumstances 

1. when TT is an infinite permutation with N as its domain and range, 

2. when tt is a finite permutation (in which case the domain and range are 
{1, 2, . . . , n} where n is the degree of tt). 

Example 2.1 Let tt be defined by: 

7r = 132654 10 987 ... 

Then Sub(7r) is easily seen to be the set of all layered permutations as defined 
inEl. 

If a = aia2 ■ ■ ■ a,„i and (3 ~ 6162 • • • are sequences (in particular, permutations) 
then their sum a® [3 is defined to be the permutation ^5 where the segments 7 
and 5 are rearrangements of 1, 2, . . . , m and m + 1, jti + 2, . . . respectively, and 
a = 7 and j3 = 5. Notice that we do not require that /3 be a finite permutation. 
If a permutation can be expressed as a® (3 (with neither summand empty) we 
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say that it is decomposable; otherwise it is said to be indecomposable. We also 
extend the sum notation to sets by defining, for any two sets of permutations 
X and Y, 

X(SY^{a®T\aeX,TeY} 

A set X of permutations is said to be sum- complete if for all a, e X, we have 
a®f]eX. 

Sum-completeness and decomposability are linked by the following result, proved 

inia. 

Lemma 2.2 Let X be a closed set with basis B. Then X is sum- complete if 
and only if B contains only indecomposable permutations. 

We shall sec that natural classes and sum-complete closed sets are closely con- 
nected. The first hint of this connection is the following result which, in partic- 
ular, shows that every sum-complete closed set is a natural class. 

Proposition 2.3 Let 7 be any (finite) permutation and S any sum-complete 
closed set. Then Sub(7) ® S is a natural class. 

Proof: Let /3i, (32, ... be any listing of the permutations of S. Consider the 
sequence of permutations 

7 ^ 7 ® /3i ^ 7 ® /3i ffi /32 ^ 7 ffi /3i © /32 © /33 ^ ■ ■ ■ 

Since S is sum-complete all these permutations lie in Sub(7) © S. On the other 
hand it is clear that every permutation of Sub(7) © 5 is involved in some term 
of the sequence. Hence Sub(7) © S satisfies condition 4 of Theorem 11.21 and 
hence is atomic. Furthermore, the proof of (4=^1) in Theorem II . 21 tells us how 
to express Sub(7) © 5* in the form Sub(7r : A ^ B). Following this recipe, it is 
easy to see that both A and B arc (isomorphic to) N, and we have a natural 
class, as required. H 

Notice that the proof of this result makes no assumption on the listing of the 
elements of S. That means that the infinite permutation tt for which Sub(7) © 
S = Sub(7r) is very far from being unique. 

In the remainder of the paper we shall be exploring a partial converse of Propo- 
sition Our main theorem will show that every finitely based natural class 
X does have the form of the proposition unless tt and X have a very particular 
form. 

3 A characterisation of natural classes 

This section is devoted to the proof of the following theorem. 
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Theorem 3.1 Let X be a finitely based natural class. Then either 

1. X = Sub(7) © S where j is a finite permutation and S is a sum-complete 
closed class determined uniquely by X , or 

2. X = Sub(7r) where tt is unique and ultimately periodic in the sense that 
there exist integers N and P > such that, for all n > N , 7r(n + P) = 
Tr{n)+P. 

The proof of the theorem will show precisely how X determines S in the first 
alternative. It will also, in the case of the second alternative, prove that X is 
enumerated by a rational generating function. 

Before embarking on a series of lemmas that lead up to the proof of Theorem 
I3.1l we shall define some notation that will be in force for the rest of this section. 

Wc shall let X = Sub(7r) where tt is a permutation of N. The basis of X will 
be denoted by B and we let b denote the length of a longest permutation in B. 
The permutations of B have a decomposition into sum components; the set of 
final components in such decompositions will be denoted by C. 

We shall use the notation A{C) for the closed set of all permutations that avoid 
the permutations of C. This is a slight extension of the notation we defined 
in Section n because C might not be the basis of A{C) [C might contain some 
non-minimal elements outside A{C)). This causes no technical difficulties. Ob- 
viously, as every permutation that avoids the permutations of C also avoids the 
permutations of B, we have A{C) C X . By Lemma [2. 21 ^fC) is sum-complete; 
it is, as we shall see, the sum-complete class S occurring in the statement of 
Theorem O 

From time to time we shall illustrate our proof with diagrams that display 
permutations. These diagrams are plots in the {x, y) plane. A permutation 
pi,P2, . ■ ■ (which maps i to Pi) will be represented by a set of points whose 
coordinates are {i,Pi). As a first use of such diagrams we have Figure Q which 
illustrates the sum operation and the two alternatives in Theorem 13. II 

Lemma 3.2 There exists an integer k such that, for all d > k, 
Sub(7r(d),7r(d+1),...) =^(C) 

Proof: For each 7 e C there is a basis element of X of the form /3 © 7. 
Every such /3 is a permutation of X and so we can choose a subsequence S{P) 
of TT with S{P) = (3. Let t be the maximal value occurring in all such S{I3) and 
let u be the right-most position of tt where an element of some S'(/3) occurs. 

There exists an integer k > u such that all terms 7r(fc + 1), 7r(fc + 2), . . . exceed t. 
Note that the order type of N is used in establishing the existence of k. Among 
the terms 7r(fc -|- 1), 7r(fc -|- 2), . . . there can be no subsequence order isomorphic 
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Figure 1: The sum of 132 and 4231 is 132 7564, as plotted on the left. Every 
finitely based natural class is defined by a finite permutation summed with a 
sum-complete class (centre), or is eventually periodic (right). 



to an element of C. This proves that Sub(7r((i), 7r((i + 1), . . .) C A{C) for all 
d > k. It also proves that .4(C) is non-empty. 

Now let 9 e .4(C). Since the permutation 1 lies in .4(C) and .4(C) is sum- 
complete we have 1, 2, . . . , c? — 1 © e .4(C). Therefore tt has a subsequence 
order isomorphic to this permutation and that implies that T:{d),TT{d + 1), . . . 
has a subsequence order isomorphic to 8 which completes the proof. ■ 

Corollary 3.3 Either X = Sub(7) © .4(C) for some finite permutation j, or 
TT has finitely many components and the last component (which is necessarily 
infinite) involves an element of C . 

Proof: Let tt = tti © 7r2 ffi • • • be the sum decomposition of tt. Lemma f3. 21 
tells us, in particular, that there is a maximal position k where a subsequence 
order isomorphic to an element of C can begin. Suppose this position occurs 
in the sum component tt^. If tt^ is not the final component of tt then we have 
Sub(7r) Sub(7ri © ■ • ■ © tt^) © Sub(7rr+i ©■••)• However 7 tti © • ■ • © tt^ is 
finite and Sub(7rr+i ©■••)= A{C) by the lemma. ■ 

The first alternative of this corollary leads to the first alternative of Theorem 
13. II because of the following uniqueness result. 

Proposition 3.4 If X = Sub(7i) (B Si ~ Sub(72) © S2 where 71,72 are finite 
permutations and Si, S2 are sum-complete then Si = 5*2- 

Proof: Let ai G 5*1. Then, as Si contains every permutation of the form 
irn = 12 ... 771, 5*1 also coutaius Lt © (Ti where t = I72I. But this permutation 
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belongs to Sub(72) ©S'2 and so can be expressed as 7'©a2 where 7' ^72- Since 
it ® CTi = 7' ffi (72 and |7'| < |tt| we have cti ^ cr2. This proves that cti e ^2 and 
therefore 5*1 C 52- The result now follows by symmetry. ■ 

In the remainder of the proof of Theorem 13.11 wc shall assume that the second 
alternative of CoroUarv 13.31 holds and work towards proving the second alterna- 
tive of the theorem. In particular, there exists a greatest position k in tt where 
a subsequence isomorphic to a permutation in C can begin, and this position 
occurs in the final (infinite) sum component tt^ of tt. 

Next we prepare the ground for two arguments that occur later in the proof and 
which depend upon the indecomposability of tt^ . Suppose that r is any position 
in TTz- Wc define a pair of sequences U{r) = U1U2 ■ ■ ■ and V{r) = V1V2 ■ ■ • by 
the following rules: 

1. Vi is the position among the terms of tt up to and including position 
(when i = 1 take Uq = r) where the greatest element occurs: 

7r(u,;) ~ max{7r(fc) | k < Ui^i}. 

2. Ui is the rightmost position in tt where a term not exceeding n^Vi) occurs: 

Ui — max{fc I 7r(fc) < n^Vi)}. 

Figure |21 depicts these points and the next lemma assures us that the figure 
accurately represents the relative positions of the marked points. 

Lemma 3.5 The relative positions and sizes of the terms 7r(wi) and 7r(wj) are 
described by the following inequalities: 

Vi < V2 < Ui < V3 < U2 < V4 < U3 < ■ ■ ■ 

7r(ui) < tt{vi) < 7r(u2) < 7r(u2) < 77(7/3) < T^i^a) < ■■ 

Proof: (I) From the definition of Vi we have Vi < Ui_i, and from the defi- 
nition of Ui we have and 7r(vi) > 7r(w.j). Note that we cannot have Ui = Ui^i-, 
because then every term of tt to the left of this position would be less than or 
equal to 7r(wi), and every term to the right would be greater than 7r(ui), contra- 
dicting the assumption that TT{vi) belongs to the final component of tt. Hence 
we have Vi < Ui-i < Ui and Tr{vi) > iT(ui). 

(II) By the definition of Wi+i, we have w^+i < Ui and 7r(wi_|_i) > 7r(wi). We 
cannot have Ui+i < Vi, because 7r(wi) is the maximal value of tt on the interval 
Also, we cannot have Wi+i — Vi, because that would imply Ui^i = u^, 
which is proved impossible as in (I). Finally, we cannot have Vi+i = Ui because 
7r(ui) < 7r(ui) by (I). We conclude that vt < Vi+i < Ui and '^{vi) < Tr{vi+i). 
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Figure 2: The terms of tt as mapped out by 7r(u,j) and 7r(u,;). All terms lie in 
the shaded boxes. 

(III) As in (I), we have li^+i > Ui and 7r(iii+i) < 7r(z;i_|_i). Moreover, u^+i > 
immediately implies that 7r(iii+i) > 7r(wi). 

(IV) As in (II), we have < Ui+i and 7r(wi+2) > 7r(ui+i). Moreover, Vi+2 > 
for otherwise we would have 7r(ui+2) < ■"'(wi+i). 

Summarising (I)-(IV), we have Vj < w^+i < < < "i+i and 7r(w,j) < 

Tr{vi) < 7r(iti+i) < 7r(ui+i) < 7r(ui+2) for every i = 1,2, . . ., which is enough to 
prove the lemma. ■ 



Our first use of the sequences U{r) and V{r) and the above lemma occurs 
immediately. We have seen fLemma l3.2f) that there is a rightmost position in tt 
where subsequences order isomorphic to permutations in C can begin. Now we 
prove that there is a rightmost position by which they have all ended. 



Lemma 3.6 There exists a position £ of n such that no subsequence of n that 
is order isomorphic to an element of C terminates after position £. 

Proof: Consider the sequences U{k), V{k) and refer to Figure El with r ^ k, 
in particular to the edge-connected strip of boxes that begins with the box Bi 
bounded by Tr(yi) and 7r(fc). Let S' be a subsequence of tt isomorphic to a 
permutation "f G C. By definition of k, S cannot start to the right of 7r(fc). In 
fact, since 7 is indecomposable, S must start in Bi, and the terms of S must 
lie in a contiguous segment of boxes. Therefore, as 1 5*1 < 6, 5 cannot extend 
beyond position U(,/2. B 

In view of this lemma we may define £ as the last position of tt that is part 
of a subsequence isomorphic to an element of C. Now we define the sequences 
U{£), V{£) (and, re- using notation, call them ui, U2, . . . and vi,V2, ■ ■ ■)■ 

The defining property of £ implies that tt ( 1 ),..., 7r(^) is the only subsequence of tt 
with this order isomorphism type. For any subsequence of tt order isomorphic to 
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TT ( 1 ),..., 7r(£) has a final element that is part of a subsequence order isomorphic 
to an element of C. Therefore this final element cannot occur after position 
£ within tt and so it must be 7r(l), . . . ,7r(£) itself. Notice also, again from the 
definition of that the permutation order isomorphic to 7r(l), . . . , 7r(£) is the 
longest permutation in X whose last element is the terminating element of a 
subsequence order isomorphic to an element of C; as such, this permutation 
depends on X and not on tt. In the next lemma we prove that a number of 
other initial segments of tt are unique of their isomorphism type, and depend 
on X rather than on tt. 

Lemma 3.7 For each i ~ 1,2,... the sequence 7r(l), . . . , 7r(wi) is the unique 
subsequence of that order isomorphism type. Its corresponding permutation 
P = /3(1), . . . , is the longest permutation in X satisfying the following 
two properties: 

(1) . . . , is isomorphic to 7r(l), . . . , 7r(Mi_i) (where uq = I ); 

(2) /3(u,) < f3{v,). 

As such, (3 depends on X only, and not on tt. 

Proof: Wc prove the lemma by induction, anchoring it at uq ~ I. Assume 
that the statements are true for some i > 1, and consider any subsequence 
7r(si), . . . ,7r(su;^j) order isomorphic to 7r(l), . . . ,7r(wi+i). By the inductive hy- 
pothesis we must have Sj = j for j = 1, . . . , w^. But then, since 7r(?ii+i) is the 
rightmost term of tt smaller than T:{vi^i ) , and since there are Wi+i — Ui — 1 terms 
between 7r(ui) and 7r(ui+i), it follows that Sj ~ j for j = Ui + 1, . . . as 
well. 

Clearly, the permutation /3 satisfies properties (1) and (2), by virtue of being 
isomorphic to 7r(l), . . . ,7r(Mi+i). Suppose that 7 = 7(1), . . . ,7(?7t,) is any per- 
mutation satisfying these conditions. Consider an embedding 7r(ti), . . . , 7r(i,„) 
of 7 in TT. As above, we must have tj = j for j = 1, . . . , Ui. And again, 7r(u,;+i) 
being the rightmost term of tt smaller than Tr(vi-\.i), we have that < Ui^i. 
But this, in turn, implies that I7I = m < tm £ Uz+i = |/^|- This proves that P is 
indeed the longest permutation of X satisfying (1) and (2). The last statement 
of the lemma is now straightforward. ■ 

At this point we can prove the uniqueness of tt: it is the limit of its initial 
segments 7r(l) • • • 7r(ui) and these depend on X alone. 

For future use we record the following result, the proof of which is analogous to 
the proof of Lemma [3. 71 

Lemma 3.8 For each i = 2,3, . . . the subsequence consisting of all terms of tt 
not exceeding TT(vi) is unique of its order isomorphism type. Its corresponding 
permutation (3 = /3(1), . . . , P{ni) is the longest permutation in X satisfying the 
following two properties: 
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(1) . . . , P{ui-i) is isomorphic to 7r(l), . . . , 7r(u,;_i); 

(2) f3{vi) is its largest term. 

As such, (3 depends on X only, and not on tt. 

The subsequences whose embeddings are unique in the previous two lemmas are 
those all of whose terms are taken from an initial contiguous strip of blocks in 
Figure 121 (with r ~ £). For convenience we let a{i) be the permutation isomor- 
phic to the first type of subsequence fLemma l3.7|l . and cr'{i) be the permutation 
isomorphic to the second type f Lemma 13. 8|l . 

Lemma 3.9 Ui+i — Ui < 2(6 — 1)^ 

Proof: Of course Ui+i — Ui is the number of terms of ■K{ui + 1) ■ ■ •7r(Mi+i). 
We divide these terms into two sets 

L = {j \ui < i < Ui+i and 7r(j) < 7r(ui+i)} 

and 

U = {j\u,<j < Ui+i and 7r(j) > ■n{v,+i)} 

and we shall show that both \L\ and \U\ arc at most [b — \Y . Each bound is 
proved in the same way and we give the details for \L\ only. Figure 13 depicts 
the locations of L and U within tt. 

Consider a maximal increasing subsequence 7r(ji)7r(j2) ■ ■ -T^ijm) of tt such that 
all ji e L. Using this we form another subsequence of tt whose terms are the 
following: 

1. all terms not exceeding 7r(wi), (a subsequence order isomorphic to a'{i)) 

2. the term irivi+i) 

3. all the terms 7r(ji)7r(j2) ■ ■ • 7r(j„) 

The permutation which is order isomorphic to this subsequence is, of course, a 
member of X and we write it as Xnp0 where n corresponds to the term '^{vi^i), 
Xp corresponds to cr'(i), and 9 ~ n — m,?i — m + l,...,7i — 1 corresponds to 
7r(ji)7r(j2) • • •7r(j„). 

Now consider another permutation almost the same as this except that 9 con- 
tains one more term, and n is replaced by n -I- 1. It has the form A, n -I- 1, p9' 
with 9' = n — m,n — m + I, . . . ,n ^ l,n. This permutation does not belong to 
X. To see this, assume that some subsequence of tt is order isomorphic to it. In 
the correspondence between the permutation and the subsequence, Xp (which 
is order isomorphic to cr'(z)) must be mapped to the subsequence of terms not 
exceeding 7r(wi), by Lemma 13.81 and n + 1 must be mapped to one of the terms 
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of 77 in the range of positions Ui-i + 1 to Ui. This forces 6' to be mapped into L 
as these are the only positions of tt to the right of Ui and smaller than 7r(w,;+i). 
However, this contradicts that L contains no increasing sequence of length m+1. 

It follows that A,n + l^pO' must involve a basis element oi X. A particular 
embedding of a basis element must contain all the terms of 0' for otherwise this 
basis element would be embedded in \np9 which is impossible. In particular we 
can deduce that m + \ <h. 

Exactly the same argument can be carried out for maximal decreasing subse- 
quences. Thus the sequence ■n{L) contains no increasing or decreasing subse- 
quence of length more than 6 — 1 and, by the well known result of Erdos and 
Szekeres (see we conclude that \L\ < (6 — 1)^. 

The proof that \U\ < {h — 1)^ is similar but it uses 7r(ui+i) and a{i) instead of 
7r(ui+i) and cr'(i). ■ 

We now define an encoding of permutations of X. If ^ = xiX2 ■ ■ ■ Xn € X then 
we encode it as E{^) = eie2 • • • e„ where 

Cfe = |{i I 1 < i < k and Xi > Xk}\ 

If a term of tt lies in the final component beyond position U2 then there are, by 
Lemma [3.91 at most 4(6 — 1)^ preceding terms greater than it. On the other 
hand, if it lies in one of the finite components or in the final component and not 
beyond position U2 then, obviously, there will again only be a bounded number 
of preceding greater terms. Since permutations of X are order isomorphic to 
subsequences of tt and tt is determined by X there is an upper bound depending 
on X alone for each of the code symbols e^. Thus we may consider E{X) as a 
language over some finite alphabet A = {1, . . . ,m}. 

" tt{v4) etc 







u 
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L ' 
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L ' 
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• 
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Figure 3: tt mapped out by Ui and Vi. None of the boxes marked L otU contains 
a monotonic increasing or decreasing subsequence of length 6. Thus the sizes of 
these boxes is bounded. 
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The encoding has the property that, if 7 = (5e is a permutation and 6 is the 
permutation order isomorphic to the initial segment S, then E{S) is an initial 
segment of -£(7). 

Furthermore this encoding (in which every element of a permutation is encoded 
by the number of its higher predecessors) is closely related to the encoding 
studied in (in which every element was encoded by its number of lower 
successors). In fact, if F(^) denotes the latter encoding and f is the permutation 
obtained from ^ by replacing each element Xi by |^| — + 1 and then reversing 
it, then F(^) is the reverse of E{^). It therefore follows from Theorem 2 of 
that 

Lemma 3.10 E(X) is a regular set. 

Wc can now confirm one of the claims we made when stating Theorem l3.1l from 
the results of the ordinary generating function of the sequence (.g„), where gn 
is the number of permutations of length n, is a rational function. But to show 
that TT is eventually periodic and thus complete the proof of the theorem we 
need to study a deterministic finite automaton that accepts E{X). We denote 
this automaton hy M = (S,v4, so,t, T) (the notation specifies, respectively, the 
set of states, the alphabet, the initial state, the transition function, and the set 
of final states). 

Lemma 3.11 {E{<T{i)) \ i — 1,2, . . .} contains the set 

I j = 0,l,2,...} 

defined by the regular expression a(3* for some non-empty words a, (3 ^ A* . 

Proof: We shall consider the sequence of states ti = T{s{),E{a{i))),i = 
1,2,... and aim to show that it is periodic. By definition and by Lemma |3.7I 
a{i + 1) = 00 where 

(a) is a sequence order isomorphic to a{i) 

(b) (j) = 0x02 ■ ■ ■ an satisfies a,i < max(0) and Ocj) £ X 

(c) (f> is maximal with these properties. 

Wc shall express these conditions in terms of the automaton M.. From (a), (b) 
and the definition of E we have 

E{cj{l + l))^E{(j{{))hih2---hn 

where w = 6162 • • ■ &n is a word in the alphabet A and riti, w) = ti+i. 

By definition, bj is the number of terms of Ocj) up to and including aj that 
exceed or equal aj. To capture the condition a„ < max(^^) we need to define 
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another sequence C0C1C2 • • • c„ where cq = and Cj is the number of terms up 
to and including Uj that exceed max(0); of course, all such terms are among 
Oi, 02, . . . , a„. 

If aj < max(0) then the terms enumerated by bj include max(0), aj and the 
Cj-i terms above max(^); hence bj > Cj_i + 1. However, if aj > max(0) then 
each of the bj terms that exceed or equal aj is one of the terms that exceeds 
max(0); hence bj < Cj_i + 1. Furthermore, in the former case Cj ~ Cj-i and 
in the latter case Cj = Cj_i + 1. Thus C1C2 ■ • - Cn is determined uniquely by 
6162 • ■ • bn, and a„ < max(0) if and only if 6„ > c„. 

Putting all this together t^+i is the unique state of A4 for which there exists a 
word w ~ bib2 • ■ • 6„ in the alphabet A with the following properties: 

1. T{ti,w) = U+i; 

2. if the sequence (cq, ci, . . . , c„) is defined by cq = and, for j > 0, 




Cj_i + 1 if bj < Cj_i + 1, 
Cj-i if bj > Cj-i + 1 



then bn > c„; 

3. w has maximal length among all words satisfying these two conditions. 

But now note that the three conditions depend only on the ti and the automaton 
A4 and not on E(a{i)). Therefore the sequence ti,t2, ... is ultimately periodic. 
So, for some P > and N we have tj ~ tj^p for all j > N. 

Let a = E{a{N)) and let P be the unique word such that E{a{N + P)) = a(3. 
Then E{a{N + hP)) = a(3^ and r(so, aP^) = tN for all h>0. This proves that 
aP* C{E(a{i))\i = l,2,...}. U 



We can now complete the proof of Theorem 13. II In the notation of the previous 
lemma, let |a| = m and \f3\ = n. Consider the encoding of tt itself: E{Tr) = 
6162 • ■ • . This is just the limit of its prefixes ^^(^(l)), E{a{2)), .... It is also 
the limit of E{a{N)), E{a{N + P)), E{a{N + 2P)), . . .. Hence, by Lemma ICT1 
E{Tr) is ultimately periodic with Cj+p ~ Cj for all j > N. 

Consider an arbitrary 7r(j) with j > N. We have 7r(j) = Ij + r j + 1 where 
Ij ^\{i\i<j and n{i) < n{j)}\ 

and 

rj = \{i\i>j and 7r(i) < 7r(j)}| 

Obviously, Ij = j — ej. The number rj can be obtained from E{Tr) as follows. 
Define two sequences v^^^ = {uq, ni, . . .) and ^^■'^ = {ho, hi, . . .) by no = and 
ho = Cj and 

_ j rij + 1 if ej+i+i > hi 
*^ I rii otherwise 
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and 

if Cj+t+i > hi 



H+l 




otherwise 



An easy inductive argument shows that rii is equal to the number of terms from 
7r(j + 1 ),..., 7r(j + i) which are smaller than 7r(j), while hi ~ Cj is the number 
of terms from the same set which are greater than In particular, v^^^ 

eventually becomes constant with value rj . 

Finally, note that z^'-'^ depends only on ejej+iej+2 ■ ■ ■ and not on 6162 ■ ■ 'ej-i. 
Hence v^^^ = v^^^-^^ and rj = Vj+p. Therefore 

7r{j + P) = lj+p+rj+p + 1 

= j + P ~ Ej+P + r-i+p + 1 
= P + j- Cj + + 1 

= P + <3) 

as required. 



4 Natural classes with infinite bases 

Any natural class class that is not of the form stipulated by the conclusion of 
Theorem 13.11 is, of course, not finitely based. An example of such a class is 

Y = Sub(7r) where 

TT = 3 2 5 1 [7, 8] 4 [10, 12] 6 [14, 17] 9 [19, 23] 13 [25, 30] 18 [32, 38] 24 . . . 

In this example [a, b] stands for the segment [a, a + 1, . . . , &]. It is clear by inspec- 
tion that TT is not periodic and so, by the uniqueness conclusion of Theorem l3.1l 

Y is not of periodic type. We argue that it is not of the form Sub(7) ® S where 
iS* is sum-complete. Suppose it were of this form. Consider the initial segments 
of TT ending with one of 1, 4, 6, 9, . . . respectively. These all define permutations 
Cii $3) ^4 • ■ ■ of y. Every is indecomposable and has a unique embedding 
in TT. From the indecomposability those of length greater than 7 must be 
order isomorphic to permutations of S] but, if G S, so also is © which 
contradicts that it is uniquely embeddable in tt. 

It would perhaps be tempting to suppose that when tt is periodic, the closed 
class Sub(7r) is always finitely based. This, however, is not the case, as our final 
example shows. 

Let X = Sub(7r), where 

TT = 2 3 5 1 7 8 4 10 6 12 13 9 15 11 ■■ ■ . 

Essentially, tt is an increasing oscillating sequence with every other left maximal 
term replaced with an increasing pair (the undcrlinings arc intended to highlight 
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ad 



Figure 4: On the left, an infinitely based periodic natural class. On the right, 
two basis elements. 



this). Call these increasing pairs twins, and note that they are the only pairs of 
terms of tt occurring in successive positions and having successive values. 

We claim that each of the following permutations belongs to the basis of X: 

Pi = 2 3 4 5 1 

P2 = 2 351748 96 

Pn = 2_3517496-- - 4n - 3 An - 6 An - I An - A An An + 1 An - 2 



The permutation /3„ is obtained from an oscillating sequence with an even num- 
ber of left maximal terms by replacing the first and last of these terms by 
increasing pairs (see Figure 01). To show that /3„ ^ X we can argue as follows. 
Write TT as 

7: = UoUihU2l2U3h • • • , 

where Uq, U2, U^, . . . are the twins, Ui, U3, U5, . . . are the remaining (unexpanded) 
left maxima, and Ii,l2,l3, ■ ■ ■ are the remaining terms. Suppose /3„ embeds into 
TT. The two twins of /3„ must correspond to two twins, say U2p and U2q, of tt. 
Since /3„ is indecomposable, the remaining left maxima 5, 7, 9, . . . of /3„ must map 
into U2P+1, U2p+2, U2P+3, ■ ■ ■ respectively. The number of left maxima between 
the two twins of /?„ is even, while the number of segments U2p+i, ■ ■ ■ , U2q-i is 
odd, a contradiction. 

To complete the proof that /3„ is a basis permutation of X, we need to demon- 
strate that Pn \ {Pn{j)}, the permutation obtained by removing the jth term 
from Pn, belongs to X for every j = 1, . . . , 4n -I- 1. If j ^ {2, 3, An, An + 1} the 
resulting permutation is decomposable, and can be embedded into tt by embed- 
ding each of its components and keeping them sufficiently apart. If j is one of 
2, 3, An or 4n -I- 1 then one of the twins of /3„ becomes a singleton. Suppose, for 
the sake of argument, that j ^ An+1 (the other cases are treated analogously). 
Then we can embed /?„ \ {/3„(4n + 1)} by mapping 2 and 3 onto Uq, all the 
other left maxima of /3„ into Ui,U2, ■ ■ ■ respectively, and the remaining terms 
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into lij2,--- respectively. Note that the parity problem which prevented us 
from embedding /3„ into tt does not arise here, because the second twin of /3„ 
has become a singleton in /3„ \ {/3„(4n + 1)}, and can therefore be mapped onto 
the singleton U2n-i- 
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